Local binary pattern (LBP) is a feature used for texture representation and classification. The LBP operator describes the surroundings of a pixel by generating a bit-code from the binary derivatives of a pixel. The operator is usually applied to greyscale images and the derivative of the intensities. Note how a pixel with a value > C is assigned ‘1’ and a pixel with a value <= C is assigned ‘0’.
#import modules and images
import numpy as np
from matplotlib import pyplot as plt
from PIL import Image
import requests
from io import BytesIO
#generate random image from lorenpicsom
#image credit: https://picsum.photos/
url = 'https://picsum.photos/224/224'
response = requests.get(url)
img = Image.open(BytesIO(response.content))
#or you can use image of mine.
img = Image.open('assets/test_image_beach.jpg')
(h,w) = img.size
img = img.resize((h//3,w//3), Image.ANTIALIAS)
img=np.asarray(img.convert("L"))
plt.imshow(img, cmap='gray', vmin=0, vmax=255)#'gist_gray')
img.shape
def lbp(img, posX, posY):
# sample matrix
# [[0 1 2]
# [3 4 5]
# [6 7 8]]
patch = img[posY-1:posY+2,posX-1:posX+2].astype(int)
# subtract values by center pixel
# [[-4 -3 -2]
# [-1 0 1]
# [ 2 3 4]]
patch = patch - patch[1,1]
# read code in circular manner
# [-4, -3, -2, 1, 4, 3, 2, -1]
patch = patch.ravel()
code = np.array([patch[0],patch[1],patch[2], patch[5], patch[8], patch[7], patch[6], patch[3]])
# clip negative to 0 and positive to 1 and reshape:
#[0 0 0 1 1 1 1 0]
code = code.clip(min=0, max=1)
#convert into int from bin
#30
code = code.dot(2**np.arange(code.size)[::-1])
return code
lbp_image = np.zeros(img.shape, dtype=int)
for x in range(1,img.shape[1]-1):
for y in range(1,img.shape[0]-1):
lbp_image[y,x] = lbp(img, x, y)
def plot(imgs, titles=[]):
fig, axs = plt.subplots(1, len(imgs), figsize=(20, 20))
i=0
for ax, theta in zip(axs, imgs):
ax.imshow(np.clip(theta,0,255), cmap='gray')
if titles: ax.title.set_text(titles[i])
i=i+1
plt.show()
plot([img,lbp_image])
import plotly.graph_objects as go
from plotly.subplots import make_subplots
fig = go.Figure(data=[go.Histogram(x=lbp_image.ravel(), histnorm='probability')])
fig.show()
Come up with a descriptor to represent the whole image as consisting of multiple windows.
Hint: Think of combining several local descriptions into a global description.
First, set the number of bins in which you want to devide the image.
Then devide the image.
#Set the bin here
bins = 5
width_window = lbp_image.shape[1] // bins
height_window = lbp_image.shape[0] // bins
windows=[]
for j in range (0, height_window*bins, height_window):
for i in range(0, width_window*bins, width_window):
windows.append(lbp_image[j:j+height_window,i:i+width_window])
#plot patches
plt.figure(figsize=(15,10))
for i, window in enumerate(windows):
plt.subplot(bins, bins, i+1)
plt.axis('off')
plt.imshow(window, cmap='gray')
plt.show()
We may get BLP codes for each patch.
fig = make_subplots(rows=bins, cols=bins, start_cell="top-left")
for i, window in enumerate(windows):
row, col=i//bins+1, i%bins+1
fig.add_trace(go.Histogram(x=window.ravel(), histnorm='probability'), row, col)
fig.update_xaxes(showticklabels=False)
fig.update_yaxes(showticklabels=False)
fig.update_layout(template="plotly_white",showlegend=False)
fig.show()
Using the global descriptor you have created, implement a classification process to classify the images in the provided dataset into two categories: face images and non-face images.
Comment the results. Hint: You can use simple histogram similarities.
Is the global descriptor able/unable to represent whole images of different types, e.g. faces vs. cars?
Identify problems (if any).
Suggest solutions if possible.
The dataset consists of a bunch of images of people’s faces taken from MIT Faces Recognition Project database.
import glob
files = glob.glob('facedata/*.pgm')
people = [np.asarray(Image.open(file).convert('L')) for file in files]
files = glob.glob('cardata/*.jpg')
cars = [np.asarray(Image.open(file).convert('L').resize((64,64))) for file in files]
plot([people[0],cars[0]])
people_lbp = np.zeros((20,64,64))
cars_lbp = np.zeros((20,64,64))
for i in range(20):
for x in range(1,63):
for y in range(1,63):
people_lbp[i,y,x] = lbp(people[i], x, y)
cars_lbp[i,y,x] = lbp(cars[i], x, y)
plot([people_lbp[0],cars_lbp[0]])
facehist=np.asarray([np.histogram(people_lbp[i,:,:], bins=255)[0] for i in range(20)])
carhist=np.asarray([np.histogram(cars_lbp[i,:,:], bins=255)[0] for i in range(20)])
faceavg = np.average(facehist, axis=0)
caravg = np.average(carhist, axis=0)
threshold = (faceavg+caravg)/2
print(threshold)
facecorrect = 0
for i in range(20):
if np.average(facehist[i]-threshold[i]) >= 12:
facecorrect += 1
carcorrect = 0
for i in range(20):
if np.average(carhist[i]-threshold[i]) < 12:
carcorrect += 1
print(facecorrect, carcorrect)
Decrease the window size and perform classification again. Comment the results.
Increase the window size and perform classification again. Comment the results.
Discuss how LBP can be used/modified for dynamic texture analysis, e.g. in a video.